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DETAILED ACTION 

1 . This is the First Office Action based on application 10/677145 filed September 
30, 2003. As originally filed, Claims 1-53 are presented for examination. 

Claim Objections 

2. Claims 1 5, 23, 25, 26, and 31 objected to because of the following informalities: 

3. Claim 1 5 recites "storing the meta-data by the a receiver." articles "the" and "a" 
can not modify the noun "receiver" at the same time (one or the other must be chosen), 
the Examiner requests "the" be deleted since this is the first time "receiver" is mentioned 
in the independent claim. 

4. Claim 23 recites the limitation "the biometric identification device" as dependent 
from Claim 20. The Examiner points out that Claim 22 provides antecedents for Claim 
23 by reciting the limitation "a biometric identification device" (not Claim 20). The 
Examiner suggests that Claim 23 be made dependent from Claim 22 in order to provide 
proper antecedent basis. 

5. In regards to Claim 25, the word "is" should be added between the words 
"sensor" and "in" to read as follows: "...the sensor is in physical contact..." 

6. Claim 26 should recite "An apparatus", not "A apparatus". 

7. Claim 31 is directed toward a method as dependent from Claim 26, which is an 
apparatus claim. The Examiner suggests that Claim 31 be changed to an apparatus 
claim in order to provide continuity in the chain of dependency. 

Appropriate correction is required. 
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Claim Rejections - 35 USC § 102 

8. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

9. Claims 1, 8, 9, 10, 11, 12, 32, 34, 35, 37, 48, 52, and 53 are rejected under 35 
U.S.C. 102(e) as being anticipated by Obrador US Patent (6,585,521 B1). 

10. In regards to Claim 1 , Obrador teaches a computerized method comprising 
receiving data representing a user's reaction to digital content (method of video indexing 
based on viewers' behavior and emotion feedback, as shown in Fig. 7 and described in 
Col. 7 Lines 22-45); and generating through computer automated operations meta-data 
based on the user's reaction (blocks 730 and 740 of Fig. 7, as described in Col. 7 Lines 
29-34). 

11. In regards to Claim 8, Obrador teaches the method of Claim 1 wherein receiving 
data representing the user's reaction further comprises receiving data representing 
nonverbal communications (image sensor 1 10 of video camera 130 "for capturing facial 
expressions and physical movement of the viewers", as disclosed in Col. 3 Lines 43-46, 
with further reference to Col. 5 Lines 64-67 and Col. 6 Lines 1-26). 

12. In regards to Claim 9, Obrador teaches the method of Claim 8 wherein the data 
representing nonverbal communications is data representing facial gestures (image 
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sensor 110 of video camera 130 "for capturing facial expressions and physical 
movement of the viewers", as disclosed in Col. 3 Lines 43-46, with further reference to 
Col. 5 Lines 64-67 and Col. 6 Lines 1-26). 

13. In regards to Claim 10, Obrador teaches the method of Claim 8 wherein the data 
representing nonverbal communications is data representing gazing patterns of the user 
(a lack of interest may be indexed if the viewer is not looking at the display, as disclosed 
in Col. 4 Lines 20-23). 

14. In regards to Claim 1 1 , Obrador teaches the method of Claim 1 wherein the 
receiving data representing the user's reaction further comprises receiving data 
representing verbal communications (audio sensor 120 may be a microphone for 
capturing an audio signal in the viewers voice, as disclosed in Col. 3 Lines 47-49, with 
further reference to Col. 6 Lines 27-67). 

1 5. In regards to Claim 12, Obrador teaches the method of Claim 1 1 wherein the 
data representing verbal communications comprises data representing speech patterns 
(emotion detection system 100 may use audio signals in the viewers' voice, such as 
laughter to categorize the video segment, as disclosed in Col. 3 Lines 31-36). 

16. In regards to Claim 32, Obrador teaches an electronic system (computer 140 of 
Fig. 2) comprising: a memory to store instructions for annotating digital content (memory 
202, as described in Col. 4 Lines 56-64); a storage device to store meta-data 
associated with the digital content (secondary storage device 212, as described in Col. 

4 Lines 57-61 ); and a processor programmed to: execute the instructions for annotating 
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digital content from the memory (processor 214, as described in Col. 4 Lines 61-64), 
annotate the meta-data with one or more user's responses to the digital content 
(emotion detection system 100 uses computer 140 to categorize different behavior and 
emotion segments of a scene, and automatically index the video sequence according to 
various categories of behavior and emotion, as described in Col. 3 Lines 56-64), and 
store the meta-data on the storage device (remote storage 150, as described in Col. 3 
Lines 60-64). 

17. In regards to Claim 34, Obrador teaches the electronic system of claim 32, 
further comprising one or more devices to collect data representing a user's response to 
digital content (emotion detection system 100 includes acquisitions devices such as, 
video cameras 130 and audio sensors 120, as described in Col. 3 Lines 37-54). 

18. In regards to Claim 35, Obrador teaches an article comprising a machine- 
accessible medium having associated data (memory 202, as described in Col. 4 Lines 
56-64), wherein the data, when accessed, results in a machine performing: receiving 
data representing a user's reaction to digital content (computer 140 executing the 
method for video indexing based on viewers' behavior and emotional feedback, as 
disclosed in Col. 4 Lines 45-48); and annotating, through computer automated 
operations, the user's reaction in meta-data associated with the digital content 
(computer 140 executing the method for video indexing based on viewers' behavior and 
emotional feedback, as disclosed in Col. 4 Lines 45-48). 
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1 9. In regards to Claim 37, Obrador teaches the article of claim 35 wherein the 
machine-accessible medium further includes data, when accessed by the machine, 
results in the machine performing transferring the meta-data to a designated location 
("computer 140 has a connection with network 160, such as the Internet or other type of 
computer or telephone networks, for sending recorded multimedia sequences and video 
indices", as disclosed in Col. 4 Lines 48-51 ). 

20. In regards to Claim 48, Obrador teaches a computerized method comprising 
automatically consolidating, from across a network, a plurality of meta-data describing a 
plurality of user's responses to digital content ("indexed multimedia sequences may be 
saved on a server to be retrieved by other users by browsing through particular 
behavior or emotional segments of multimedia", as disclosed in Col. 5 Lines 56-63). 

21 . In regards to Claim 52, Obrador teaches the computerized method of claim 48 
further comprising analyzing verbal communication delivered in the form of digital 
content, the analyzing performed using user responses to the digital content annotated 
in the meta-data (audio sensor 120 may be a microphone for capturing an audio signal 
in the viewers voice, as disclosed in Col. 3 Lines 47-49, with further reference to Col. 6 
Lines 27-67. In addition, emotion detection system 100 can categorize various 
categories of audio feedback within a video sequence 680, such as the "speech" 
segment 620 of Fig. 6, as disclosed in Col. 7 Lines 4-13). 

22. In regards to Claim 53, Obrador teaches a the computerized method of Claim 48 
wherein the consolidated meta-data provides an automatic ethnographic ranking system 
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for digital content (Obrador discloses "video indices of behavior and emotion feedback 
may be labeled, for example, as "happy #1", "funny #2", or "interesting #4", within the 
video sequence for easy indexing" from Col. 6 Lines 23-26. In addition, indexing may 
be done in a hierarchical representation, as disclosed in Col. 6 Lines 46-48). 



Claim Rejections - 35 USC § 103 

23. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

24. Claims 2, 3, 1 5, 1 6, 1 7, 1 8, 21 , 26, 27, 28, 29, 31 , 33, 36, 45, 46, and 47 are 
rejected under 35 U.S.C. 103(a) as being unpatentable over Obrador US Patent 
(6,585,521 B1) in view of Lu et al US Patent (5,771,307) hereinafter "Lu." 

25. In regards to Claim 2, Obrador teaches the method of Claim 1 , but does not 
teach the method further comprising receiving data identifying a user. 

In a similar field of invention, Lu teaches a method for identifying a user based on 
aspects such as, body shape matching, viewer tracking, and/or historical data. Lu's 
method establishes a reference facial image, which is used as reference to compare to 
a current image. When the images are determined to be the same, the user is uniquely 
identified. 
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It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user monitoring method of Obrador with the user 
identification method of Lu because "if the individual members of a sample household 
are not uniquely identified, the various demographic information usually provided in 
viewing reports is not, generally speaking, readily ascertainable" (as disclosed by Lu in 
Col. 2 Lines 43-46). Therefore, by uniquely identifying a user in the monitoring system 
a more accurate and personalized profile is established. 

26. In regards to Claim 3, Obrador and Lu teach the method of Claim 2, further 
comprising generating a ranking of one or more items of digital content based on the 
reaction of one or more users to the digital content (Obrador discloses "video indices of 
behavior and emotion feedback may be labeled, for example, as "happy #1", "funny #2", 
or "interesting #4", within the video sequence for easy indexing" from Col. 6 Lines 23- 

26. In addition, indexing may be done in a hierarchical representation, as disclosed in 
Col. 6 Lines 46-48). 

27. In regards to Claim 15, Obrador teaches a method comprising collecting the 
user's reactions to digital content (method of video indexing based on viewers' behavior 
and emotion feedback, as shown in Fig. 7 and described in Col. 7 Lines 22-45); 
generating meta-data associated with the digital content based on the user's reactions 
(blocks 730 and 740 of Fig. 7, as described in Col. 7 Lines 29-34); and storing the meta- 
data by a receiver (indexed sequences may be transmitted to the network 160 and/or 
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the remote storage 150 to be stored, as disclosed in Col. 3 Lines 56-62). Obrador does 
not teach the limitation of identifying a user of digital content. 

In a similar field of invention, Lu teaches a method for identifying a user based on 
aspects such as, body shape matching, viewer tracking, and/or historical data. Lu's 
method establishes a reference facial image, which is used as reference to compare to 
a current image. When the images are determined to be the same, the user is uniquely 
identified (Lu Abstract). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user monitoring method of Obrador with the user 
identification method of Lu because "if the individual members of a sample household 
are not uniquely identified, the various demographic information usually provided in 
viewing reports is not, generally speaking, readily ascertainable" (as disclosed by Lu in 
Col. 2 Lines 43-46). Therefore, by uniquely identifying a user in the monitoring system 
a more accurate and personalized profile is established. 

28. In regards to Claim 16, Obrador and Lu teach the method of Claim 15 further 
comprising transferring the meta-data from the receiver to an originator (Obrador 
discloses that indexed video sequences may be transferred to network 160 or remote 
storage 150, as disclosed in Col. 4 Lines 56-60. In addition, network 160 may be "the 
Internet or other type of computer or telephone networks, for sending recorded 
multimedia sequence and video indices to friends and family by, for example, email", as 
disclosed in Col. 4 Lines 48-52). 
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29. In regards to Claim 17, Obrador and Lu teach the method of Claim 15 further 
comprising transferring the meta-data from the receiver to a location identified by the 
originator (Obrador discloses in Col. 4 Lines 48-52: network 160 may be "the Internet or 
other type of computer or telephone networks, for sending recorded multimedia 
sequence and video indices to friends and family by, for example, email." Therefore, 
using Obrador's example of email, a user may inherently address the email with 
attached multimedia content to any other user with a valid email address). 

30. In regards to Claims 18 and 21 , Onrador and Lu teach the method of Claim 15 
wherein identifying a user is performed using an electronic identification devices (Lu 
teaches a method for identifying a user based on aspects such as, body shape 
matching, viewer tracking, and/or historical data. Lu's method establishes a reference 
facial image, which is used as reference to compare to a current image. When the 
images are determined to be the same, the user is uniquely identified. Image capturing 
is preformed by video cameras 32 and 34, as described in Col. 9 Lines 1-20). 
Additionally, Onrador and Lu teach that the electronic identification device is remote 
from the user (Lu's method teaches video cameras 32 and 34 are located in video 
equipment module 18, as disclosed in Col. 9 Lines 21-23. In addition, from Fig. 1 , it can 
be seen that video equipment module 18 is on top of television set 12 and clearly at 
remote location from users 24 and 26). 

31. In regards to Claim 26, Obrador teaches an apparatus comprising: a mechanism 
to collect the user's responses to the digital content (emotional detection system 100 of 
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Fig. 1 A, as described in Col. 3 Lines 37-64); a mechanism to generate a plurality of 
meta-data associated with the digital content based on the user's responses (emotion 
detection system 100 may combined facial expressions and the viewer's voice to 
categorize a video segment, the categorized video segment may then be used to index 
the video sequence, as disclosed in Col. 4 Lines 9-28); and a mechanism to transfer the 
plurality of meta-data generated throughout a network to a location identified by an 
originator (network 1 60 may be "the Internet or other type of computer or telephone 
networks, for sending recorded multimedia sequence and video indices to friends and 
family by, for example, email." as disclosed in Col. 4 Lines 48-52. Therefore, using 
Obrador's example of email, a user may inherently address the email with attached 
multimedia content to any other user with a valid email address). Obrador does not 
teach a mechanism to identify a user of digital content. 

In a similar field of invention, Lu teaches a video equipment module 18, as 
described in Col. 9 Lines 1-40, for identifying a user based on aspects such as, body 
shape matching, viewer tracking, and/or historical data. Computing system 52 
establishes a reference facial image, which is used as reference to compare to a current 
image. When the images are determined to be the same, the user is uniquely identified 
(see functional bock diagram of Fig. 3, with further reference to Col. 10 Lines 22-59). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the emotional detection system of Obrador with the video 
equipment module of Lu because "if the individual members of a sample household are 
not uniquely identified, the various demographic information usually provided in viewing 
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reports is not, generally speaking, readily ascertainable" (as disclosed by Lu in Col. 2 
Lines 43-46). Therefore, by uniquely identifying a user in the monitoring system a more 
accurate and personalized profile is established. 

32. In regards to Claim 27, Obrador and Lu teach the apparatus of Claim 26 wherein 
the location identified is the location of the originator (Obrador discloses that indexed 
video sequences may be transferred to network 1 60 or remote storage 1 50, as 
disclosed in Col. 4 Lines 56-60. In addition, network 160 may be "the Internet or other 
type of computer or telephone networks, for sending recorded multimedia sequence and 
video indices to friends and family by, for example, email", as disclosed in Col. 4 Lines 
48-52). 

33. In regards to Claim 28, Obrador and Lu teach the apparatus of Claim 26 wherein 
the mechanism to identify a user further comprises one or more electronic identification 
devices (Lu teaches a video equipment module 18, as described in Col. 9 Lines 1-40, 
for identifying a user based on aspects such as, body shape matching, viewer tracking, 
and/or historical data. The images used to identify the user are acquired using cameras 
32 and 34, as described in Col. 9 Lines 1-20). 

34. In regards to Claim 29, Obrador and Lu teach the apparatus of Claim 26 wherein 
the mechanism to identify a user further comprises one or more biometric identification 
devices (Lu teaches a video equipment module 18, as described in Col. 9 Lines 1-40, 
for identifying a user based on aspects such as, body shape matching, viewer tracking, 
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and/or historical data. The images used to identify the user are acquired using cameras 
32 and 34, as described in Col. 9 Lines 1-20). 

35. In regards to Claim 31 , Obrador and Lu teach the apparatus of Claim 26 wherein 
the mechanism to collect the user* response is a device selected from the group 
consisting of a keyboard, a mouse, a remote control, a touchpad, a joystick, a speech 
recognition device, a video camera, and a microphone (Lu teaches a video equipment 
module 18, as described in Col. 9 Lines 1-40, for identifying a user based on aspects 
such as, body shape matching, viewer tracking, and/or historical data. The images 
used to identify the user are acquired using cameras 32 and 34, as described in Col. 9 
Lines 1-20). 

36. In regards to Claim 33, Obrador and Lu teach the electronic system of Claim 32, 
further comprising one or more devices for identifying a user of digital content (Lu 
teaches a video equipment module 18, as described in Col. 9 Lines 1-40, for identifying 
a user based on aspects such as, body shape matching, viewer tracking, and/or 
historical data. The images used to identify the user are acquired using cameras 32 
and 34, as described in Col. 9 Lines 1-20). 

37. In regards to Claim 36, Obrador and Lu teach the article of Claim 35 wherein the 
machine-accessible medium further includes data, wherein the data, when accessed by 
the machine, results in the machine performing identifying the user (functional block 



Application/Control Number: 10/677,145 Page 14 

Art Unit: 2623 

diagram of Fig. 2 is performed by computing system 52, as disclosed by Lu in Col. 9 
Lines 41-47). 

38. In regards to Claim 45, Obrador teaches a system comprising means to collect 
the user's reactions to digital content (emotional detection system 100 of Fig. 1A, as 
described in Col. 3 Lines 37-64); means to generate meta-data associated with the 
digital content based on the user's responses/reactions (emotion detection system 100 
may combined facial expressions and the viewer's voice to categorize a video segment, 
the categorized video segment may then be used to index the video sequence, as 
disclosed in Col. 4 Lines 9-28); and means to consolidate in a local database the 
multiple meta-data generated throughout the network ("indexed multimedia sequences 
may be saved on a server to be retrieved by other users by browsing through particular 
behavior or emotional segments of multimedia", as disclosed in Col. 5 Lines 56-63). 
Obrador does not teach a means to identify a user of digital content. 

In a similar field of invention, Lu teaches a video equipment module 18, as 
described in CoL 9 Lines 1-40, for identifying a user based on aspects such as, body 
shape matching, viewer tracking, and/or historical data. Computing system 52 
establishes a reference facial image, which is used as reference to compare to a current 
image. When the images are determined to be the same, the user is uniquely identified 
(see functional bock diagram of Fig. 3, with further reference to Col. 10 Lines 22-59). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the emotional detection system of Obrador with the video 
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equipment module of Lu because "if the individual members of a sample household are 
not uniquely identified, the various demographic information usually provided in viewing 
reports is not, generally speaking, readily ascertainable" (as disclosed by Lu in Col. 2 
Lines 43-46). Therefore, by uniquely identifying a user in the monitoring system a more 
accurate and personalized profile is established. 

39. In regards to Claim 46, Obrador and Lu teach the system of Claim 45 further 
comprising means to provide a ranking based on the consolidated meta-data (Obrador 
discloses "video indices of behavior and emotion feedback may be labeled, for example, 
as "happy #1", "funny #2", or "interesting #4", within the video sequence for easy 
indexing" from Col. 6 Lines 23-26. In addition, indexing may be done in a hierarchical 
representation, as disclosed in Col. 6 Lines 46-48). 

40. In regards to Claim 47, Obrador and Lu teach the system of Claim 45 further 
comprising means to browse selected items of digital content based on the meta-data 
("indexed multimedia sequences may be saved on a server to be retrieved by other 
users by browsing through particular behavior or emotional segments of multimedia", as 
disclosed by Obrador in Col. 5 Lines 56-63). 

41 . Claims 4, 5, 6, 7, and 13 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Obrador US Patent (6,585,521 B1 ) in view of Cliff et al US Patent 
(6,885,304) hereinafter "Cliff." 
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42. In regards to Claim 4, Obrador teaches the method of Claim 1 , but does not 
teach wherein the data representing the user's reaction is data representing an 
individual's reaction. 

In a similar field of invention, Cliff teaches a method of monitoring the reaction of 
people to a performance using two or more monitoring devices of different types. In 
addition, Cliff teaches a method of "providing one or more first monitoring devices each 
adapted to provide at least one output signal form which the behavior of an individual 
reacting to the performance can be estimated" (as disclosed in Col. 2 Lines 54-57). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user monitoring method of Obrador with the method of 
obtaining an individuals reaction to a performance, as taught by Cliff, because if an 
environment provides multiple rooms, each with a different performance occurring, it 
would be advantageous to identify the reaction of an individual user so that, if multiple 
users are experiencing different performances, an accurate reaction for each individual 
would be established. 

43. In regards to Claim 5, Obrador teaches the method of Claim 1 , but does not 
teach wherein the data representing the user's reaction is data representing a group's 
reaction. 

In a similar field of invention, Cliff teaches a method of monitoring the reaction of 
people to a performance using two or more monitoring devices of different types. In 
addition, Cliff teaches a method of "providing at least one output signal form which the 
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behavior of a plurality of those people reacting to the performance can be estimated" 
(as disclosed in Col. 2 Lines 58-61). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user monitoring method of Obrador with the method of 
obtaining a group reaction to a performance, as taught by Cliff, because the overall 
reaction of a group of individuals would represent an approximation of the average 
reaction and therefore be a more effective representation of the individual reactions of 
multiple individuals, who may have different reactions to the same performance. 
44. In regards to Claims 6 and 7, Obrador teaches the method of Claim 1 , but does 
not teach wherein receiving data representing a user's reaction further comprises 
receiving data representing physiological processes, selected from a group consisting of 
breathing, heart rate, blood pressure, galvanic response, eye movement, and muscle 
activity. 

In a similar field of invention, Cliff teaches a method of monitoring users' reaction 
to a performance, which includes monitoring a users' body temperature, perspiration, 
and pulse rate (as disclosed in Col. 6 Lines 6-24). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user monitoring method of Obrador with the method of 
obtaining a physiological reaction to a performance, as taught by Cliff, because 
"monitoring of these features can also... allow central determination of whether 
audience members are at risk of endangering their health" (as disclosed in Col. 6 Lines 
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25-28). Monitoring the physiological processes of a user would therefore insure the 
safety and wellbeing of the user. 

45. In regards to Claim 1 3, Obtador teaches the method of Claim 1 1 , but does not 
teach wherein the data representing verbal communications comprise data representing 
the user's vocabulary. 

In a similar field of invention, Cliff teaches a method of detecting one or more 
keywords spoken by the user. In Cliffs method, a negative response signal may be 
produced for one keyword and a positive signal for a different keyword (as disclosed in 
Col. 5 Lines 55-62). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user monitoring method of Obrador with the method of 
obtaining a performance based on a keyword spoken by an individual, as taught by Cliff, 
because the spoken word would provide a specific reaction of the user and therefore a 
more accurate interpretation of the user's feelings about the performance can be 
established. 

46. Claims 14, 50, and 51 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Obrador US Patent (6,585,521 B1 ) in view of Eldering et al US 
Patent (7,150,030 B1) hereinafter "Eldering." 

47. In regards to Claim 14, Obtador teaches the method of Claim 1 , but does not 
teach wherein receiving data representing the user's reaction further comprises 
receiving data representing the user's pattern used to browse the digital content. 
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In a similar field of invention, Eldering teaches a method of monitoring a user's 
actions, such as the time duration programming is watched and the volume at which the 
programming is listened. These user inputs are then used to create program 
characteristics vectors, which can be used in combination with subscriber selection data 
to form a subscriber profile (Eldering Abstract). Further more, Eldering's method keeps 
track of the number of channel changes occurring during the viewing time using a 
channel change record (as disclosed in Col. 7 Lines 44-47). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user monitoring method of Obrador with the method of user 
input tracking, such as channel change patterns, as taught by Eldering because it is well 
known in the art that rapid channel changing, or channel surfing, signifies the searching 
(or browsing) through programs that are not of interest. Therefore, by tracking the input 
of a user as taught by Eldering, a determination that a user is channel surfing may be 
established. 

48. In regards to Claim 50, Obrador teaches the computerized method of Claim 48, 
but does not teach the method further comprising adjusting a price for the digital content 
in response to the user responses to the digital content annotated in the meta-data. 

In a similar field of invention, Eldering teaches a method of monitoring a user's 
actions to create a user profile. Eldering also teaches providing the user with a discount 
on a product to service if the user shares their profile with, for instance, an advertiser. 
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The advertiser can then provide content that is tailored to the likes and dislikes of the 
user, based on the user profile (as Eldering discloses in Col. 2 Lines 35-44). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user monitoring method of Obrador with the method of 
providing a discount to a user as taught by Eldering because providing the user with a 
discount for products and services would increase the likelihood of the user to disclose 
personal information in the profile by providing an incentive. 

49. In regards to Claim 51 , Obrador teaches the computerized method of Claim 48, 
but does not teach the method further comprising conducting market research using the 
user responses to the digital content annotated in the meta-data. 

In a similar field of invention, Eldering teaches a method of monitoring a user's 
actions to create a user profile. Eldering also teaches using average and session 
household demographic characteristics as an indication of advertisements the 
household would be interested in watching. Household interest column 1701 of Fig. 17 
specifies the types of programming and products which are statistically characterized for 
a given household (as Eldering discloses in Col. 13 Lines 53-67). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user monitoring method of Obrador with the method of 
program and product characterization as taught by Eldering because this data would 
indicate "what types of products cold potentially be advertised with a high probability of 
the advertisement being watched in its entirety" (as disclosed by Eldering in Col. 13 
Lines 53-62). 
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50. Claims 19, 20, 22, 23, 24, 25, and 30 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Obrador US Patent (6,585,521 B1) in view of Lu et al US 
Patent (5,771 ,307) hereinafter "Lu" in further view of Cliff et al US Patent (6,885,304) 
hereinafter "Cliff." 

51. In regards to Claims 19, and 20, Obrador and Lu teach the method of Claims 15 
and 18 wherein identifying a user is performed using an electronic identification device 
(see Claims 15 and 18 for references regarding these limitations). Obrador and Lu do 
not teach wherein the electronic identification device is worn or carried by the user. 

In a similar field of invention, Cliff teaches a method of monitoring the reaction of 
people to a performance using two or more monitoring devices of different types. One 
of these devices is depicted in Fig. 1. In Col. 4 Lines 21-31 and Col. 5 Lines 10-12 Cliff 
discloses that device 1 may also function as a wristwatch, which is therefore worn by 
the user. In addition, Cliff discloses in Col. 5 Lines 29-37 that the method of monitoring 
may also be embodied in a handheld computing device, such as a PDA, which would 
therefore be carried by the user. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user identification and user reaction method, as taught by 
Obrador and Lu, with the method of user identification using a portable device, as 
taught by Cliff, because a portable device in close proximity to the user would provide a 
an indirect user response by monitoring the users state of well being or physical 
behavior, as described by Cliff in Col. 5 Lines 41-45). 
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52. In regards to Claims 22 and 23, Obrador, Lu, and Cliff teach the method of claim 
20 wherein the biometric identification device is selected from the group consisting of a 
fingerprinting device, a voice recognition device, an iris pattern identification device, a 
retinal pattern identification device, a face recognition device, and a key stroke rhythm 
detection device (Lu teaches a method for identifying a user based on aspects such as, 
body shape matching, viewer tracking, and/or historical data. Lu's method establishes a 
reference facial image, which is used as reference to compare to a current image. 
When the images are determined to be the same, the user is uniquely identified, see Lu 
Abstract.). 

53. In regards to Claim 24 and 25, Obrador and Lu teach the method of Claim 1 5, 
but do not teach wherein collecting the user's reactions is performed using a sensor or 
wherein the sensor is in physical contact with the user. 

In a similar field of invention, Cliff teaches a method of monitoring the reaction of 
people to a performance using two or more monitoring devices of different types. One 
of these devices is depicted in Fig. 1. In Col. 4 Lines 21-31 and Col. 5 Lines 10-12 Cliff 
discloses that device 1 may also function as a wristwatch, which is therefore worn by 
the user. In addition, Cliff teaches that device 1 may have a number of sensors, such 
as a temperature, perspiration, and pulse, as disclosed in Col. 6 Lines 6-26. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user identification and user reaction method, as taught by 
Obrador and Lu, with the method of monitoring a user by way of a sensor, as taught by 
Cliff, because a device in close proximity to the user would provide a user response by 
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monitoring the users state of well being or physical behavior, which would require little 
effort from the user, as described by Cliff in Col. 5 Lines 41-45). 

54. In regards to Claim 30, Obrador and Lu teach the apparatus of Claim 26, but do 
not teach wherein the mechanism to collect the user's response further comprises one 
or more sensors. 

In a similar field of invention, Cliff teaches an apparatus for monitoring the 
reaction of people to a performance using two or more monitoring devices of different 
types, one of these devices is depicted in Fig. 1 . In addition, Cliff teaches that device 1 
may have a number of sensors, such as a temperature, perspiration, and pulse, as 
disclosed in Col. 6 Lines 6-26. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user identification and user reaction method, as taught by 
Obrador and Lu, with the method of user identification using a portable device that 
consists of a biometric identification device, as taught by Cliff, because a portable 
device in close proximity to the user would provide user response by monitoring the 
users state of well being or physical behavior, which would require little effort from the 
user, as described by Cliff in Col. 5 Lines 41-45). 



i 
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55. Claims 38, 40, 41 , and 42 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Obrador US Patent (6,585,521 B1) in view of Vetro et al US Patent 
Application Publication (2003/0156108 A1). 

56. In regards to Claim 38, 40, and 41 , Obrador teaches an article comprising a 
machine-accessible medium having associated data representing digital content and 
meta-data associated with the digital content wherein the meta-data comprises one or 
more annotations representing a user's response to the digital content (operations of 
computer 140, as described in Col. 3 Lines 56-64). Obrador does not teach a data 
structure comprising a first field containing data representing digital content, and a 
second field containing data representing meta-data associated with the digital content. 
Nor does Obrador teach wherein the data representing the digital content is a Digital 
Item or wherein the data representing meta-data is stored using a schema based on 
MPEG-21. 

In a similar field of invention, Vetro teaches, as prior art, a digital item 100 as part 
of the MPEG-21 framework, which includes one or more resources 110, associated 
descriptors 120, and structure 130, as shown in Fig. 1 . Vetro further discloses that 
resources 110 can include individual multimedia assets 111 (such as MPEG video) and 
that descriptors 120 include descriptive information about the internals of the resource 
(such as an MPEG-7 description), as disclosed in Paragraph [0006]. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the article for indexing media as taught by Obrador with the data 
structure in the form of a MPEG-21 Digital Item as taught by Vetro because, as 
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disclosed by Vetro, "One fundamental advantage of the digital item 100 is that it can 
aggregate multiple different resources into a single logical unit, or package, with a rich 
set of descriptive information" as stated in Paragraph [0007]. In addition, Vetro 
discloses "A second fundamental advantage of the digital item is that it is self-describing 
and configurable" as stated in Paragraph [0008]. A digital item with a resource and a 
descriptor, as described by Vetro, would therefore provide a versatile data structure that 
could easily be identified by a variety of devices. 

57. In regards to Claim 42, Obrador and Vetro teach the article of Claim 38 wherein 
the machine-accessible media further includes data, wherein the data comprises a data 
structure for use by the machine, wherein the annotation identifies an event (Obrador 
teaches: operations of computer 140, as described in Col. 3 Lines 56-64; with further 
reference to Col. 7 Lines 4-21). 

58. Claim 39 is rejected under 35 U.S.C. 103(a) as being unpatentable over Obrador 
US Patent (6,585,521 B1) in view of Vetro et al US Patent Application Publication 
(2003/0156108 A1) in further view of "Reconciling MPEG-7 and MPEG-21 Semantics 
through a Common Event-Aware Metadata Model" by Hunter. 

59. In regards to Claim 39, Obrador and Vetor teach the article of Claim 38, but do 
not teach wherein the machine-accessible media further includes data, wherein the data 
comprises a data structure for use by the machine, the data structure comprising a third 
field containing data representing an originator of the digital content. 
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It is well know in the art that meta-data may be represented using the MPEG-7 
standard. Hunter describes the Usage description tool of MPEG-7, which "describe[s] 
when, where, how, and by whom the multimedia content has been used" (as disclosed 
in the second paragraph of Hunter's Introduction). The Examiner interprets the "where" 
of this section to be equivalent to Applicants "originator" of media content. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the article for indexing media as taught by Obrador using Vetro's 
digital item data structure (comprising MPEG-7 descriptors) because Hunter shows that 
MPEG-7 data structures inherently have the capability of tracking location or origin data 
using the Usage description tool. 

60. Claims 43 and 44 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Obrador US Patent (6,585,521 B1 ) in view of Vetro et al US Patent Application 
Publication (2003/0156108 A1) in further view Eldering et al. US Patent (7,150,030 B1) 
hereinafter "Eldering." 

61 . In regards to Claims 43 and 44, Obrador and Vetor teach the article of Claim 38, 
but do not teach wherein the machine-accessible media further includes data, wherein 
the data comprises a data structure for use by the machine, wherein the data structure 
representing meta-data comprises responses from multiple users or wherein the data 
representing meta-data comprises statistical summaries of response from multiple 
users. 
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In a similar field of invention, Eldering teaches a subscriber characterization 
system that uses heuristic rules or conditional probabilities to aid in the generation of an 
interest profile. A household interest profile 180, as shown in Fig. 17, is generated 
using process 1600 of Fig. 16, as described in Col. 13 Lines 41-67). As shown in Fig. 
17, the average value 1703 and session value 1705 of various household interests are 
generated using the process of Fig. 16. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the article for indexing media as taught by Obrador using Vetro's 
digital item data structure with the subscriber characterization system of Eldering 
because, as Eldering discloses "the heuristic rules in logical form allow the system to 
apply generalizations which have been learned from external studies to obtain a 
characterization of the subscriber" from Col. 2 Lines 24-27. Constructing a subscriber 
characterization system using conditional probabilities, as disclosed by Eldering, would 
therefore provide a general user profile representing the likes and dislikes of the 
majority of users. 



62. Claim 49 is rejected under 35 U.S.C. 103(a) as being unpatentable over Obrador 
US Patent (6,585,521 B1) in view Schlack et al. US Patent (7,260,823 B2). 
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63. In regards to Claim 49, Obrador teaches the computerized method of Claim 48, 
but does not teach wherein consolidating the meta-data s performed when a network is 
idle. 

In a similar field of invention, Schlack teaches a viewer characterization and 
profiling system based on the viewer's interactivity with a television, which is used to 
create a user profile initially stored in STB [Set-Top Box] 220. Schlack's method 
involves aggregating user data from each STB in order to form groups of viewers with 
similar characteristics. Schlack further discloses in Col. 36 Lines 55-60 that "the 
signature profiles stored in STB 220 may be transmitted to the HE [Head End] at 
determined in intervals (i.e. every night at off-peak hours, every week)." 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combined the user monitoring method of Obrador with the method of data 
aggregation during off-peak hours, as taught by Schlack because there is a high 
likelihood that network traffic will be at a minimum during off-peak hours of operation. 
Performing the aggregation process during off-peak hours would therefore increase the 
available bandwidth on the network during peak usage hours. 

Conclusion 

64. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure, as listed below. 
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65. Zawilinski US Patent (5,676,138) teaches a multimedia computerized system for 
detecting the emotional responses of a viewer and the changes in response over time. 
In addition, Zawilinski teaches a number of measuring devices for measuring and 
recording a plurality of values associated with each of a plurality of physiological 
variables. Statistical calculations are used to generate a descriptor for an associated 
emotional response. 

66. Dimitrova et al. US Patent Application Publication (2003/0093784 A1 ) teach a 
television audience feedback collection system and a method for using human response 
inputs to automatically create a viewer profile. Dimitrova uses a number of viewer 
feedback sensors in order to provide an unobtrusive and automatic method of gauging 
audience reaction to audio-visual content. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to PATRICK A. RYAN whose telephone number is 
(571 )270-5086. The examiner can normally be reached on Mon to Thur, 8:00am - 
5:00pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Scott Beliveau can be reached on (571) 272-7343. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

IP. A. R.I 

Examiner, Art Unit 2623 
Thursday, February 21 , 2008 
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